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Operational  Use  of  the  Air  Traffic 
Selection  and  Training  Battery 


A  Plan  for  the  Future:  The  FAA’s  10-Year  Strategy  for 
the  Air  Traffc  Control  Workforce  was  submitted  to  the 
U.S.  Congress  in  December  2004.  This  report,  generally 
referred  to  as  the  Controller  Workforce  Plan  (FAA,  2004), 
provides  the  outline  to  mitigate  anticipated  controller 
retirements  and  contemplates  strategies  to  achieve  ap¬ 
propriate  staffing  levels,  including  hiring  12,500  con¬ 
trollers  over  the  next  decade.  The  present  paper  focuses 
on  the  current  status  of  the  recently1  implemented  Air 
Traffic  Selection  and  Training  (AT-SAT)  selection  test 
battery,  focusing  specifically  on  the  functioning  of  the 
reweighted  AT-SAT  and  the  potential  for  group  differ¬ 
ences  that  could  result  in  adverse  impact.2  The  data  in 
this  paper  represent  the  first  time  operational  (collected 
for  selection  purposes  rather  than  research)  AT-SAT  data 
are  being  reported. 

How  did  the  staffing  situation  become  so  urgent? 
As  detailed  in  the  Controller  Workforce  Plan,  a  majority 
of  the  air  traffic  controller  workforce  went  on  strike  on 
August  3,  1981,  when  President  Ronald  Reagan  ordered 
the  striking  controllers  to  return  to  duty  within  48  hours. 
When  10,438  (out  of  a  workforce  of  approximately 
15,000)  striking  controllers  did  not  return  to  work  by 
the  deadline,  President  Reagan  fired  them.  Facing  a 
sudden  shortage  of  controllers,  the  Federal  Aviation 
Administration  (FAA)  hired  3,416  individuals  in  1982 
and  another  1,720  in  1983.  From  1982  through  1991, 
the  FAA  hired  an  average  of  1,527  individuals  per  year. 
The  majority  of  entrants  met  the  18  to  30  years-of-age 
entry  requirement. 

The  post-strike  hiring  wave  created  the  potential  for  a 
large  portion  of  the  controller  workforce  to  reach  retire¬ 
ment  age  at  roughly  the  same  time,  particularly  due  to 
the  FAA  policy  requiring  retirement  from  controlling  air 
traffic  by  age  56.  Based  on  current  projections,  73%  of  the 
agency’s  15,000  controllers  will  become  eligible  to  retire 
within  ten  years.  Total  losses  are  expected  to  reach  nearly 
11,000  (FAA,  2005).  This  amount  of  attrition  means 
that  the  FAA  must  use  effective  recruitment,  selection, 


'AT-SAT  was  approved  as  the  official  ATCS  selection  test  for  those 
applicants  without  previous  air  traffic  control  experience  on  May 
13,  2002,  with  June  2002  marking  the  first  time  the  test  was  used 
operationally. 

2Adverse  Impact  -  “A  selection  rate  for  any  race,  sex,  or  ethnic  group 
which  is  less  than  4/ 5  (80%)  of  the  rate  for  the  group  with  the  highest 
rate.”  (Uniform  Guidelines  on  Employee  Selection  Procedures,  1978, 
Sec  4D.) 


and  training  procedures  to  ensure  that  its  staffing  needs 
are  met.  The  AT-SAT  battery,  now  the  official  Civil  Ser¬ 
vice  test  used  to  select  FAA  air  traffic  control  specialists 
(ATCSs)  without  previous  operational  air  traffic  control 
experience,  will  thus  become  an  instrument  of  increasing 
importance. 

The  development  and  validation  of  AT-SAT  plays  a 
critical  role  in  reducing  costs  associated  with  attrition 
from  air  traffic  control  training.  Using  a  valid  selection 
test  also  ensures  that  those  who  are  hired  have  (or  have 
the  potential  to  develop)  the  necessary  knowledge,  skills, 
and  abilities  to  be  successful.  The  duties  of  ATCSs  make 
them  individually  responsible  for  more  lives  than  the 
practitioners  of  any  other  occupation  in  the  United  States 
(Biggs,  1979).  The  FAA  developed  the  AT-SAT  battery 
to  replace  a  two-stage  selection  process  in  which  ATCS 
applicants  completed  an  Office  of  Personnel  Management 
(OPM)  test  battery  and  a  nine-week  screening  program  at 
the  FAA  Academy  in  Oklahoma  City,  OK.  This  previous 
selection  process  proved  to  be  expensive  (Ramos,  Heil,  & 
Manning,  2001a).  AT-SAT  was  developed  based  on  the 
results  of  the  Separation  and  Control  Hiring  Assessment 
(SACHA;  Nickles,  Bobko,  Blair,  Sands,  &Tartak,  1995) 
job  task  analysis,  which  drew  heavily  from  previous  work 
done  by  Ammerman,  Becker,  Jones,  Tobey,  and  Phillips 
(1987).  Additional  development  and  validation  efforts  for 
AT-SAT  were  described  by  Ramos,  Heil,  and  Manning 
(2001a  &  2001b)  in  their  two-volume  report. 

AT-SAT  is  a  select-in  procedure;  select-out  (medical) 
issues  will  not  be  addressed  in  this  paper.  Readers  interested 
in  select-out  issues  are  referred  to  King,  Retzlaff,  Detwiler, 
Schroeder,  and  Broach  (2003).  AT-SAT  is  a  computerized 
test  battery  comprised  of  eight  subtests  based  on  22  indi¬ 
vidual  scores  that,  when  weighted  (forming  “part  scores”) 
and  combined,  are  totaled  (with  a  constant  added)  for  a 
single  overall  score.  As  delineated  in  Table  1 ,  AT-SAT  is 
comprised  of  the  following  subtests:  Air  Traffc  Scenarios 
Test,  ATST;  Analogies,  AY;  Angles,  AN;  Applied  Math,  AM; 
Dials,  DP,  Experiences  Questionnaire,  EQ,  Letter  Factory, 
LF;  and  Scan,  SC.  AT-SAT  is  an  aptitude  test  and  not 
a  test  of  air  traffic  control  knowledge.  The  goal  of  AT- 
SAT  is  to  predict  the  likelihood  of  success  in  air  traffic 
control  training  and,  more  importantly,  subsequently 
on  the  job.  Seven  of  the  eight  subtests  assess  aspects  of 
cognitive  ability,  while  one,  EQ,  assesses  issues  in  the 
personal  history/personality  realm.  Four  [ATST,  AY,  LF, 
SC)  of  the  subtests  are  dynamic;  they  are  interactive  and 
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Table  1.  The  eight  AT-SAT  subtests. 


Subtest 

Description 

Dials  (DI) 

Scan  and  interpret  readings  from  a  cluster  of  analog 
instruments 

Applied  Math  (AM) 

Solve  basic  math  problems  as  applied  to  distance,  rate,  and 
time 

Scan  (SC) 

Scan  dynamic  digital  displays  to  detect  targets  that  regularly 
change 

Angles  (AN) 

Determine  the  angle  of  intersecting  lines 

Letter  Factory  (LF) 

Participate  in  an  interactive  dynamic  exercise  that  requires 
categorization  skills,  decision  making,  prioritization,  working 
memory  (incidental  learning),  and  situation  awareness 

Air  Traffic  Scenarios  (ATST) 

Control  traffic  in  interactive,  dynamic  low-fidelity  simulations 
of  air  traffic  situations  requiring  prioritization 

Analogies  (AY) 

Solve  verbal  and  nonverbal  analogies  that  require  working 
memory  and  the  ability  to  conceptualize  relationships 

Experience  Questionnaire  (EQ) 

Respond  to  Likert  scale  questionnaire  about  life  experiences 

can  only  be  administered  via  computer.  The  remaining 
four  are  static,  similar  to  pencil-and-paper  tests,  but  are 
administered  via  computer. 

During  the  development  of  AT-SAT,  subtests  were 
weighted  to  yield  the  maximum  validity,  according  to 
their  relationship  to  the  job,  as  suggested  by  the  job  task 
analysis.  An  individual’s  performance  on  each  subtest  is 
multiplied  by  the  weight  developed  for  that  subtest,  and 
a  constant  is  subtracted  to  ensure  that  each  subtest  has  a 
floor  of  zero.  The  net  result  is  a  “part  score.”  Part  scores 
are  then  summed  and  combined  with  an  additional, 
overall  constant  (composed  of  a  summation  of  the  inverse 
of  the  individual  constants  and  a  calibrating  constant) 
to  yield  the  overall  AT-SAT  score  (which  is  truncated 
to  have  a  maximum  score  of  100).  Veteran’s  preference 
points  (either  five  or  ten  points)  are  subsequently  added. 
This  overall  score  is  the  only  one  that  enters  into  the 
hiring  decision  when  AT-SAT  is  used  in  an  operational 
fashion.  Subtest  weights  are  not  disclosed  in  this  paper 
as  they  could  be  used  as  part  of  a  coaching  strategy  to 
artificially  inflate  AT-SAT  scores  in  an  effort  to  gain  a 
competitive  advantage. 

Before  operational  use  of  AT-SAT  was  approved  for 
hiring  purposes,  FAA  employees  that  were  members  of 
minority  groups  raised  concerns  over  potential  adverse 
impact.  Recall  that  adverse  impact  is  a  selection  rate 
for  any  race,  sex,  or  ethnic  group  that  is  less  than  4/5 
(80%)  of  the  rate  for  the  group  with  the  highest  rate.  To 
calculate  this  ratio,  the  pass  rate  of  the  group  of  interest 


serves  as  the  numerator,  while  the  pass  rate  of  the  group 
with  the  highest  pass  rate  serves  as  the  denominator. 
Eighty  percent  serves  as  the  bright  line;  a  quotient  be¬ 
low  80%  suggests  a  potential  for  adverse  impact,  while 
one  equal  to  or  greater  than  80%  argues  against  it.  The 
concern  about  the  potential  for  adverse  impact  against 
African  Americans  seemed  well  founded,  as  only  three 
out  of  every  100  black  applicants  were  predicted  to 
achieve  a  score  of  at  least  70  (the  minimum  passing  score 
-  termed  a  “qualifying  score”)  on  AT-SAT.  Nevertheless, 
as  the  predictive  validity  was  .69,  a  case  could  be  made 
for  “business  necessity.”  Business  necessity  is  a  defense 
available  when  the  employer  has  a  criterion  for  selection 
that  is  facially  neutral  but  excludes  members  of  one  sex, 
race,  or  national  origin  at  a  substantially  higher  rate  than 
members  of  other  groups  (thus  creating  adverse  impact). 
The  employer  must  then  prove  that  its  selection  require¬ 
ment  having  the  adverse  impact  is  job-related,  typically 
as  demonstrated  by  a  job  analysis. 

To  appreciate  the  wider  context,  the  reader  needs  to 
understand  that  the  issue  went  beyond  pass  rates  for 
minority  applicants.  By  original  design,  38%  of  fully 
certified  incumbent  FAA  controllers  would  not  pass 
AT-SAT  under  the  original  scoring  scheme.  The  original 
passing  score  of  70  had  been  calibrated,  using  the  overall 
constant,  so  that  only  62%  of  incumbent  fully  certified 
controllers  would  achieve  an  AT-SAT  score  of  70  or 
more,  in  an  effort  to  minimize  FAA  Academy  failures 
and  compensate  for  the  need  for  ATCSs  to  perform 
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potentially  more  difficult  duties  in  the  future.  The  goal 
was  to  at  least  preserve  and  strive  to  improve  the  level  of 
functioning  in  the  workforce  (Waugh,  2001). 

In  response  to  concerns  about  group  differences  that 
could  result  in  adverse  impact,  FAA  officials  requested 
that  scientists  review  the  options  to  mitigate  these  dif¬ 
ferences.  Meanwhile,  they  emphasized  maintaining  the 
overall  validity  of  the  AT-SAT  battery.  Additionally,  FAA 
management  made  the  case  that  the  cut  score  should  be 
set  at  the  point  where  most  fully  qualified,  incumbent 
controllers  would  pass  FAA’s  entry-level  aptitude  test. 
Consequently,  the  AT-SAT  subtests  were  re-weighted, 
and  the  overall  constant  was  adjusted.  The  new  weights 
were  developed  as  a  trade-off  between  their  validity  and 
their  contribution  to  group  differences  that  could  lead  to 
adverse  impact.  The  content  of  the  subtests  themselves  was 
not  changed;  rather,  the  subtests  were  weighted  differently. 
The  goal  was  to  retain  adequate  validity  while  reducing 
potential  adverse  impact.  Test  validity  (job-relatedness) 
is  determined  by  the  strength  of  the  correlation  between 
the  overall  AT-SAT  score  and  en  route  controller  job 
performance  measures  (which,  in  the  AT-SAT  validation 
study,  consisted  of  a  computerized  situational  judgment 
test  assessing  maximum  performance,  plus  peer  and  super¬ 
visor  ratings  of  typical  performance).  After  reweighting, 
the  correlation  between  AT-SAT  and  a  job  performance 
composite  measure  was  reduced  slightly,  from  .69  to  .60 
(Wise,  Tsacoumis,  Waugh,  et  al.,  2001).  Compared  with 
most  validation  coefficients,  this  is  still  a  strong  correlation 
with  job  performance.  As  indicated  above,  the  relation¬ 
ship  with  job  performance  is  especially  important  in  this 
context;  any  remaining  adverse  impact  can  be  justified  by 
business  necessity,  as  previously  defined  above. 

Using  data  from  the  original  validation  studies,  Wise 
et  al.  (2001)  found  that,  by  reweighting  the  subtests 
and  adjusting  the  overall  constant  (as  described  above), 
they  eliminated  group  differences  that  could  result  in 
potential  adverse  impact  for  women  and  Hispanics  and 
greatly  reduced  it  for  African  Americans.  Wise,  et  al. 
concluded  their  report  with  a  cautionary  statement  about 
the  uncertainty  of  how  the  reweighting  might  function 
with  actual  applicants,  to  include  the  impact  on  overall 
pass  rates.  A  primary  purpose  of  the  present  paper  is  to 
examine  how  this  reweighting  functioned  operation¬ 
ally  to  eliminate  group  differences  that  could  result  in 
adverse  impact. 

To  further  address  the  potential  problem  of  adverse 
impact,  FAA  officials  decided  to  abandon  a  strict  “top- 
down”  approach  to  hiring  and,  instead,  use  a  category 
ranking  method.  This  approach  is  a  form  of  “score  band¬ 
ing”  that  can  be  justified  on  the  basis  of  ignoring  score 
differences  that  are  due  to  an  estimate  of  the  applicant’s  true 
ability.  Score  banding,  although  somewhat  controversial 


among  selection  scientists,  “will  almost  always  produce 
less  adverse  impact  than  strict  rank  ordering”  (Biddle, 
2005,  p.  103)  as  it  ignores  score  differences  likely  to  be 
statistically  insignificant.  Under  this  scheme,  job  fair 
applicants  who  achieve  a  qualifying  minimum  score  are 
divided  into  two  groups:  those  scoring  85  and  above 
(termed  “well  qualified”)  and  those  scoring  from  70  to 
84.9  (termed  “qualified”).  Those  in  the  “well-qualified” 
group  will  be  offered  employment  before  anyone  in  the 
“qualified”  group.  AT-SAT  data  in  this  paper  are  therefore 
reported  according  to  these  categories. 

Dattel  and  King  (2006)  applied  the  weights  and  ad¬ 
ditive  constant  developed  to  address  potential  adverse 
impact  to  the  scores  of  724  developmental  ATCSs,  hired 
by  a  previous  method,  who  volunteered  to  take  AT-SAT. 
An  average  increase  of  4.86  points  was  found  with  the 
new  scoring  method;  the  notional  passing  rate,  based  on 
achieving  an  AT-SAT  score  of  70  or  more,  changed  from 
58.8%  to  80%.  American  Indian/Alaskan  Native,  His¬ 
panic,  and  black  participants  (categories  are  as  defined  on 
the  Race  and  National  Origin  Form)  showed  the  greatest 
average  increase  in  overall  scores:  6.97,  6.98,  and  7.02 
respectively,  representing  increases  in  pass  rates  of  22.2% 
(77.8%  to  100%),  35.1%  (51.9%  to  87%),  and  35.2% 
(37%  to  72.2%),  respectively. 

While  this  analysis  of  data  collected  from  research 
participants  was  encouraging,  the  real  standard  is  to 
determine  how  a  selection  instrument  functions  with 
actual  applicants.  That  opportunity  has  presented  itself 
with  the  commencement  of  increased  hiring  and  universal 
use  of  AT-SAT  to  hire  candidates  without  previous  air 
traffic  control  experience. 

Sources  of  ATCS  Applicants  without  Previous  Controlling 
Experience 

The  Air  Traffic  -  Collegiate  Training  Initiative  (AT 

-  CTI)  program  and  job  fairs  are  two  authorities  used  to 
hire  personnel  without  previous  experience  controlling  air 
traffic.  The  AT  -  CTI  program  is  a  partnership  between 
the  FAA  and  14  aviation  colleges/universities  to  provide 
the  academic  preparation  necessary  for  students  interested 
in  ATCS  careers.  Enrollment  in,  and  even  completion  of, 
a  AT  -  CTI  program  in  no  way  obligates  the  FAA  to  hire 
the  student,  and  the  FAA  typically  does  not  test  these 
students  until  near  the  completion  of  their  studies.  AT 

-  CTI  school  officials  decide  whether  to  recommend  a 
student  to  the  FAA  for  AT-SAT  testing,  and  hence,  hir¬ 
ing  consideration. 

Not  all  job  fair  applicants  are  permitted  to  take  the 
AT-SAT  due  to  the  time  required  (a  full  day)  and  the 
cost  (about  $800)  of  administering  it.  Selection  for  AT- 
SAT  testing  was  previously  based  on  a  random  selection 
process  but  is  now  based  on  responses  to  a  biographical 
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questionnaire.  Job  fair  applicants  compete  with  each 
other  in  order  to  be  hired  at  a  specific  facility.  Not  every 
job  fair  applicant  who  achieves  a  qualifying  score  will  be 
offered  employment.  Consequently,  it  is  possible  that 
a  job  fair  applicant  who  scores  only  in  the  “qualified” 
range  will  not  be  offered  employment.  Similarly,  AT 
-  CTI  applicants  who  score  in  the  “well  qualified”  range 
are  referred  for  positions  before  applicants  who  score  in 
the  “qualified”  range. 

The  third  pool  of  applicants  also  did  not  have  previ¬ 
ous  air  traffic  control  experience  but  were  unique  in  that 
they  took  both  the  paper  and  pencil  Office  of  Personnel 
Management  (OPM)  test  and  AT-SAT.  Members  of  this 
group  had  passed  the  OPM  test  several  years  previously 
but  were  not  admitted  into  the  nine-week  screening  pro¬ 
gram  due  to  reduced  staffing  requirements  at  the  time. 
Since  hiring  was  reduced  after  they  passed  the  OPM  test, 
but  before  they  could  complete  the  screening  program, 
they  had  to  take  AT-SAT  at  a  later  time  and  could  not  be 
hired  without  a  passing  AT-SAT  score  (>70).  This  group 
is  also  unique  due  to  the  amount  of  time  that  had  elapsed 
before  they  were  finally  considered  to  be  hired  and  their 
two  tiers  of  testing. 

Other  Applicant  Pools 

There  are  other  applicant  pools  for  ATCS  positions, 
including  former  military  and  Department  of  Defense 
civilian  controllers.  These  groups  will  not  be  described  in 
detail  in  this  paper  as  they  do  not  have  to  take  AT-SAT 
and  are  considered  for  employment  based  on  their  previ¬ 
ous  operational  experience  controlling  air  traffic. 


METHOD 

Operational  results  (i.e.,  AT-SAT  scores  collected  from 
applicants  who  competed  on  the  basis  of  taking  AT-SAT) 
were  analyzed  in  terms  of  race,  ethnic,  and  gender  group 
membership,  considered  by  hiring  authority. 

Participants 

From  June  2002  to  June  2006,  854  applicants  took 
AT-SAT  as  part  of  their  j  ob  application  process .  Applicants 
either  1)  responded  to  a  job  fair  announcement  (solicit¬ 
ing  applicants  for  a  specific  position),  2)  were  nearing 
completion  of  one  of  the  14  AT  -  CTI  programs,  or  3) 
had  previously  passed  the  OPM  test  and  had  to  achieve 
a  passing  score  on  AT-SAT  before  they  were  admitted 
into  training  at  the  FAA  Academy.  All  applicants  were 
requested  to  voluntarily  complete  the  Race  and  National 
Origin  (RNO)  form  (OPM  Form  1468  until  October 
2005,  FAA  Form  3330-64  thereafter)  to  ascertain  their 
racial/ethnic  group  membership.  Self-identification  of 
gender  was  new  to  FAA  Form  3330-64;  previously,  that 
information  was  solicited  through  a  variety  of  means.  Of 
854  job  applicants,  gender  was  known  for  253  (29.63%). 
RNO  data  are  discussed  in  the  Results  section,  as  they 
derive  directly  from  the  self-report  forms. 

RESULTS 

Of  the  8  54  j  ob  applicants ,  2 1 9  (2 5 . 64%)  disclosed  their 
race  on  the  form  in  use  at  the  time  of  their  application. 
Total  counts  for  race/national  origin  groups  and  gender 
are  depicted  in  Table  2. 


Table  2.  RNO  Results. 


RNO 

Count 

1.)  American  Indian  or  Alaska  Native 

1 

2.)  Asian  (Asian  or  Pacific  Islander  previous  to  Oct  2005) 

12 

3.)  Black  or  African  American  (Black,  not  of  Hispanic  Origin  previous  to  Oct  05) 

23 

4.)  Hispanic  or  Latino  (Hispanic  previous  to  Oct  05) 

8 

5.)  Native  Hawaiian  or  Other  Pacific  Islander  (Was  represented  under  #2  previous  to  Oct  05) 

1 

6.)  White 

174 

Gender 

Male 

212 

Female 

41 
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Table  3  displays  AT- SAT  pass  rates  (those  who  achieved 
a  score  >70)  with  respect  to  self-reported  RNO  group 
membership,  segmented  by  hiring  authority. 

Collapsing  across  applicant  pools,  the  sole  American 
Indian/Alaska  Native  applicant  passed  and  the  sole  Na¬ 
tive  Hawaiian/Other  Pacific  Islander  passed;  11  of  12 
(91.67%)  Asian  applicants  passed;  18  of  23  (78.26%) 
black  applicants  passed;  7  of  8  (87.5%)  Hispanic  appli¬ 
cants  passed;  and  169  of  174  (97.13%)  white  applicants 
passed.  Ignoring  the  two  smallest  groups  (due  to  their 
extremely  small  n),  the  ratio  of  the  group  with  the  lowest 
passing  rate  (black/ African  American)  compared  to  the 
group  with  the  highest  passing  rate  is  80. 57  (78.26/97. 1 3) , 
just  above  the  80%  threshold,  suggesting  that  AT-SAT 
did  not,  as  yet,  exhibit  group  differences  that  could  result 
in  adverse  impact. 


As  the  concept  of  adverse  impact  is  also  concerned  with 
gender  differences,  the  data  were  also  examined  by  gender, 
again  according  to  hiring  authority.  Table  4  displays  AT- 
SAT  pass  rates  (those  who  achieved  a  score  >70)  with 
respect  to  gender,  segmented  by  hiring  authority. 

Collapsing  applicant  pools,  203  of  2 1 2  men  (95.75%) 
passed  and  38  of  41  women  (92.68%)  passed.  The  ratio 
here  is  96.79  (92.68/95.75)  suggesting  that  AT-SAT 
did  not  exhibit  a  gender  difference  with  a  potential  for 
adverse  impact. 

Table  5  delineates  the  results  of  the  overall  population 
of  854,  by  hiring  source.  The  overall  pass  rate  of  93.3% 
rate  is  higher  than  the  67%  predicted  in  the  Controller 
Workforce  Plan  (FAA,  2004). 


Table  3.  RNO  Group  AT-SAT  differences,  considered  by  hiring  source. 


Self-reported  RNO 

Applicant 

Failed 

scored 

Qualified  - 

Well  Qualified  - 

Total  Passing  - 

Membership 

Pool 

below  70 

scored  70-85 

scored  >  85 

>70 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

1 .)  American  Indian  or 
Alaska  Native 

OPM/ATSAT 

1 

100.0% 

1 

100.0% 

2.)  Asian  (Asian  or 

CTI 

2 

66.7% 

1 

33.3% 

3 

100.0% 

Pacific  Islander 

Job  Fair 

1 

14.3% 

3 

42.9% 

3 

42.9% 

6 

85.7% 

previous  to  Oct  2005) 

OPM/ATSAT 

2 

100.0% 

2 

100.0% 

3.)  Black  or  African 

CTI 

2 

14.3% 

6 

42.9% 

6 

42.9% 

12 

85.7% 

American  (Black,  not  of 
Hispanic  Origin 
previous  to  Oct  05) 

Job  Fair 

OPM/ATSAT 

3 

42.9% 

4 

2 

57.1% 

100.0% 

4 

2 

57.1% 

100.0% 

4.)  Hispanic  or  Latino 

CTI 

1 

20.0% 

4 

80.0% 

4 

80.0% 

(Hispanic  previous  to 

Oct  05) 

Job  Fair 

1 

33.3% 

2 

66.7% 

3 

100.0% 

5.)  Native  Hawaiian  or 
Other  Pacific  Islander 
(Was  represented  under 
#2  previous  to  Oct  05) 

Job  Fair 

1 

100.0% 

1 

100.0% 

CTI 

3 

3.9% 

20 

26.0% 

54 

70.1% 

74 

96.1% 

6.)  White 

Job  Fair 

14 

41.2% 

20 

58.8% 

34 

100.0% 

OPM/ATSAT 

2 

3.2% 

20 

31.7% 

41 

65.1% 

61 

96.8% 

Total 

12 

5.5% 

76 

34.7% 

131 

59.8% 

207 

94.5% 

Table  4.  Gender  AT-SAT  differences,  considered  by  hiring  source. 


Gender 

Applicant  Pool 

Failed  -  scored 
below  70 

Qualified  - 
scored  70-85 

Well  Qualified  - 
scored  >  85 

Total  Passing  - 
>70 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

CTI 

4 

4.5% 

30 

34.1% 

54 

61.4% 

84 

95.5% 

Male 

Job  Fair 

2 

4.8% 

16 

38.1% 

24 

57.1% 

40 

95.2% 

OPM/ATSAT 

3 

3.7% 

19 

23.2% 

60 

73.2% 

79 

96.3% 

CTI 

3 

13.6% 

6 

27.3% 

13 

59.1% 

19 

86.4% 

Female 

Job  Fair 

3 

75.0% 

1 

25.0% 

4 

100.0% 

OPM/ATSAT 

7 

46.7% 

8 

53.3% 

15 

100.0% 

Total 

12 

4.7% 

81 

32.0% 

160 

63.3% 

241 

95.3% 
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Table  5.  Overall  AT-SAT  results,  considered  by  hiring  source. 


Applicant  Pool 

Failed  -  scored  below 
70 

Qualified  -  scored 
70-85 

Well  Qualified  - 
scored  >  85 

Total  Passing  - 
>  70 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

Count 

Row  % 

CTI 

34 

6.9% 

178 

35.9% 

284 

57.3% 

462 

93.1% 

Job  Fair 

20 

7.7% 

109 

41.8% 

132 

50.6% 

241 

92.3% 

OPM/ATSAT 

3 

3.2% 

26 

26.8% 

68 

70.1% 

94 

96.9% 

Total 

57 

6.7% 

313 

36.7% 

484 

56.7% 

797 

93.3% 

DISCUSSION  AND  CONCLUSION 

After  years  of  validation  efforts  (Ramos,  Heil,  &  Man¬ 
ning,  2001a  &  2001b),  AT-SAT  is  finally  being  used  as 
an  operational  selection  tool  for  ATCSs,  coinciding  with 
the  increased  hiring  that  has  already  begun.  This  study 
focused  primarily  on  AT-SAT’s  performance  in  selecting 
applicants  of  various  racial/ethnic  backgrounds.  The 
outcome  found  in  this  study  is  very  different  than  the 
3%  passing  rate  predicted  under  the  original  weighting 
scheme  for  black  applicants,  for  example.  There  was 
also  no  evidence  of  any  potential  adverse  impact  with 
respect  to  gender.  Therefore,  while  AT-SAT  appears 
to  be  functioning  as  projected  by  Wise  et  al.  (2001)  in 
terms  of  reduced  group  differences  that  could  result  in 
adverse  impact,  the  higher  than  expected  pass  rates  may 
be  of  concern.  The  next  step  is  to  conduct  longitudinal 
validation  to  determine  how  well  AT-SAT  predicts  success 
in  training  and  on-the-job  performance. 

The  increased  ATCS  hiring  will  serve  to  populate  the 
RNO  and  gender  cells  if  applicants  voluntarily  provide 
the  data.  Group  differences  that  could  result  in  adverse 
impact  will  be  continually  monitored.  The  value  of  con¬ 
sidering  the  data  by  hiring  source  is  to  suggest  strategies 
to  mitigate  group  differences,  should  they  arise.  Recall 
that  AT-SAT  scores  are  used  differently  depending  on 
the  hiring  authority  under  which  an  applicant  applies. 
Therefore,  even  though  the  present  study  is  more  realis¬ 
tic  than  previous  efforts  that  used  research  data,  a  more 
thorough  understanding  could  be  afforded  by  an  examina¬ 
tion  of  applicants  who  are  actually  offered  employment. 
In  an  effort  to  increase  the  voluntary  participation  rate, 
future  applicants  should  be  encouraged  to  complete 
their  RNO  forms  so  that  a  more  accurate  picture  of  the 
applicants  can  be  achieved  and  group  differences  can  be 
more  readily  detected.  It  will  also  be  necessary  to  assess 
the  potential  for  adverse  impact  of  subsequent  selection 
procedures,  to  include  considering  who  is  actually  offered 
employment. 
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